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ABSTRACT 



A study examined the effect of participation in a tech prep 
program, specifically an applied communications course, on the 
problem-solving self -appraisal and critical thinking skills of secondary 
students. It also explored effects of a traditional English course and an 
honors English course on these factors. A pretest-posttest nonequivalent 
control group design was used. The Problem Solving Inventory Form B (Heppner 
1988) measured the individual's self -appraisal of problem-solving 
self-efficacy; the Watson-Glaser Critical Thinking Appraisal Form B (Watson 
and Glaser 1980) measured critical thinking abilities. The sample consisted 
of students enrolled in four sections of three English courses: applied 
communications, traditional English, and honors English. Complete data were 
obtained from 140 students. The Statistical Analysis System was used to 
calculate a multivariate analysis of variance with the general linear model 
adaptation for a two- factor repeated measures experiment or a Pearson 
Product -Moment correlation, as appropriate. Findings indicated the following: 
a statistically significant difference in mean total scores on both tests by 
type of English course; no statistically significant difference in mean total 
scores by time of test administration; and a correlation for posttest scores 
significantly different than zero. Critical thinking abilities of students 
who completed the applied communications course did not significantly improve 
and their perceptions of their problem-solving skills did not become more 
positive. (Contains 16 references.) ( YLB) 
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Employers and educators are generally in agreement that students need to increase 
their problem solving and critical thinking skills. Numerous national reports have pointed 
out that students are deficient in this area. The Tech Prep initiative has tended to focus on 
activities designed to better prepare students for the world of work (Custer, Ruhland, & 
Stewart, 1997). 

Various approaches have been proposed for improving the problem solving and 
critical thinking skills of students. Three approaches found in the literature were the 
general, infusion, and immersion methods which Sormunen (1992) discussed. The general 
method of teaching critical thinking is by a separate course outside of any content area. 

The infusion method consists of teaching general principles of reasoning within the context 
of subject matter. The immersion method focuses on the knowledge content without 
pointing out general critical thinking principles. 

The theoretical base of this study was formed from cognitive and behavioral 
learning theories and theories of self-efficacy as applied to problem solving and critical 
thinking. These learning theories can be categorized as theories that fall within two broad 
areas. The first area deals with the nature of problem solving and critical thinking. The 
second with whether problem solving and critical thinking can be taught, and how to teach 
it if it can be taught. 

The ability to think critically and to solve problems has been a concern of 
philosophers, educators, and psychologists for many centuries. Sternberg (1986) stated that 
Plato and Aristotle founded the critical thinking movement. These philosophers were 
concerned with the first area of theory. Sternberg attributes the modem-day movement to 
John Dewey. Dewey (1933) was concerned with the nature and value of thinking. He 
considered thinking to be the process by which individuals find meaning in the world in 
which they live. The ability to think critically is a prerequisite for problem solving and as 
such is of significant value. Dewey believed that the ability to think critically, or 
reflectively, was a function of one's experience as well as one's intellect. 

The basis for the second area of learning theory, that which deals with methods for 
developing cognitive abilities, can be traced to Bloom (1956). Bloom developed a 
taxonomy of cognitive levels of learning. Each level requires a different mental activity or 
way of thinking. Lower levels of learning are a prerequisite for higher levels, where higher 
levels are often referred to as requiring higher-order thinking skills. Problem solving 
requires higher-order thinking, which according to Bloom can be taught. 

A number of authors have proposed methods for teaching thinking and problem 
solving. For example, Bruner, Goodnow, & Austin, (1956), Gallagher (1993), Halpem 
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(1984), and Ruggiero (1988) have described various aspects of thinking, learning, and 
problem solving and methods for developing problem solving skills. The methods they 
proposed are based upon both cognitive and behavioral learning theories. 

The basis for theories of self-efficacy can be attributed to Bandura (Greenberg and 
Baron, 1993). They discussed the development of self-efficacy. The theory is that self- 
efficacy develops "partly through direct experiences, in which individuals perform various 
tasks and receive feedback on their success, and partly through vicarious experiences, in 
which they observe others performing various tasks and attaining varying levels of success 
at them" (p. 208). They further stated that, regardless of how it develops, “the stronger 
individuals’ beliefs that they can perform successfully, the higher their performance 
actually tends to be” (p. 208). 

Problem solving is one of the foundation skills needed for high-skill, high-wage 
employment (United States Department of Labor, 1992; Daggett, 1992). Life in 
technologically oriented countries, such as the United States, is characterized by rapid 
change. This rapid change complicates life and makes it necessary for students to learn 
how to be effective problem solvers. It is particularly desirable that students learn to be 
effective problem solvers in the context of actual production in work situations. In order to 
keep up with international competition and the faster pace of technological change, 
employers in the United States are demanding that the public schools be responsible for 
developing the critical thinking and problem solving skills of students. In response to this 
demand, goals have been defined at both the national and state level for developing 
students’ problem solving skills. 

The fifth objective of National Education Goal 6 calls for increasing the proportion 
of college graduates who demonstrate the ability to think critically and solve problems. 
Specifically, the objective is that: “The proportion of college graduates who demonstrate 
an advanced ability to think critically, communicate effectively, and solve problems will 
increase substantially” (National Education Goals Panel, 1994, p. 10). 

This study investigated the extent to which the development of problem solving 
self-efficacy and critical thinking skills, based upon learning theory and theory explaining 
the development of self-efficacy, were accomplished in a Tech Prep applied 
communications course. 

Purpose 

The primary purpose of this study was to examine the effect of participation in a 
Tech Prep program and specifically an applied communications course on the problem 
solving self-appraisal and selected aspects of critical thinking skills of secondary students. 
A secondary purpose was to examine the effects of a traditional English course and an 
honors English course on these factors. The study was conducted to answer the following 
questions: 

1 . Is there a significant difference in the mean total scores from a self-appraisal of 
problem solving abilities and from a measurement of critical thinking skills among 
students completing an applied communications course, a traditional English course, or 
an honors English course? 



2. Is there a significant difference in students’ self-appraisal of their problem solving 
abilities and in selected aspects of students’ critical thinking skills when measured near 
the beginning and near the end of an applied communications course, a traditional 
English course, or an honors English course? 

3. Is there a significant relationship between the students’ self-appraisal of their problem 
solving abilities and their critical thinking skills for students completing an applied 
communications course, a traditional English course, or an honors English course? 

Methods 



A pretest-posttest nonequivalent control group design was used for the study. 
Demographic information and two instruments were used to collect data. 

The instrument used to measure the individual’s self appraisal of their problem 
solving self-efficacy for this study was the Problem Solving Inventory (PSI) Form B 
(Heppner, 1988). The PSI contains 35 items and uses a 1 to 6 Likert-type scale with 1 
representing strong agreement and 6 strong disagreement with the statement. Reliability 
for the PSI was reported to be r =.89 for test-retest reliability and r =.90 for internal 
consistency. The instrument used to measure selected aspects of the students’ critical 
thinking abilities was the Watson-Glaser Critical Thinking Appraisal (WGCTA) Form B 
(Watson and Glaser, 1980). The instrument consists of 80 items divided into five 
subscales of 16 items each and includes exercises which are purported to be examples of 
problems, statements, and interpretations of data which are regularly encountered at work 
or school. The split-half reliability estimates for eleventh grade students was reported to be 
r =.79. The maximum raw score for the WGCTA is 80. 

The purposive sample consisted of the students enrolled in four sections of each of 
three different English courses. The three courses were comprised of an applied 
communications course, a traditional English course, and an honors English course. Where 
there were more than four sections of a course, the sections for this study were randomly 
selected. The total number of students enrolled in the 12 sections included in this study 
was 254 at the time of the pretest, increasing to 279 by the time of the posttest. Complete 
data, which consisted of pretest and posttest measurements for both the PSI and WGCTA, 
were obtained from 136 students. These 136 students included 53 Honors English in 
students, 43 English HI students, and 40 applied communications students. All of these 
students were in the 1 1th grade, with the exception of one 10th grade student who was 
assigned to English in, and three 12th grade students who were assigned to applied 
communications. 

The Statistical Analysis System (SAS) was used to calculate a MANOVA 
(multivariate analysis of variance ) with the general linear model adaptation for a two 
factor repeated measures experiment or a Pearson Product-Moment correlation, as 
appropriate. The null hypotheses were rejected if the F value was significant at equal to or 
less than the .05 alpha level. 

Findings and Conclusions 

The mean pretest and posttest scores (Table 1) on the PSI for the Honors English 
students were 83.7 and 79.0; for the English HI students were 92.4 and 90.8; and for the 
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applied communications students were 93.2 and 95.7. The mean pretest and posttest scores 
on the WGCTA for the Honors English students were 57.5 and 59.1; for the English III 
students were 48.5 and 45.6; and for the applied communication students were 43.1 and 
42.6. 

Table 1 

Mean Pretest and Posttest Scores for the PSI and WGCTA 



Variable 


M 


n 


SD 


Variance 


SE 


Course 1 -Honors English IE 
Pretest PSI 


83.736 


53 


17.811 


317.237 


2.447 


Posttest PSI 


79.000 


53 


19.104 


364.962 


2.624 


Pretest WGCTA 


57.453 


53 


6.256 


39.137 


.859 


Posttest WGCTA 


59.151 


53 


7.140 


50.977 


.981 


Course 2-English IE 
Pretest PSI 


92.419 


43 


14.945 


223.344 


2.279 


Posttest PSI 


90.791 


43 


18.809 


353.788 


2.868 


Pretest WGCTA 


48.488 


43 


7.830 


61.303 


1.194 


Posttest WGCTA 


45.628 


43 


8.449 


71.382 


1.288 


Course 3-English IE C 
Pretest PSI 


93.225 


40 


19.453 


378.435 


3.076 


Posttest PSI 


95.725 


40 


21.664 


469.333 


3.425 


Pretest WGCTA 


43.100 


40 


7.063 


49.887 


1.117 


Posttest WGCTA 


42.575 


40 


6.898 


47.584 


1.091 



The first hypothesis, HOI, stated there is no statistically significant difference in 
the mean total scores from the PSI and WGCTA by type of English course assignment. 
HOI was rejected at the .05 level of significance. As reported in Tables 2 and 3, there was 
a significant difference in mean total scores of students by English course assignment. 
Students assigned to the honors English HI course had mean total scores which were 
significantly more positive than the mean total scores of students assigned to English m. 
The mean total scores of English IE students were, in turn, significantly more positive than 
the mean total score of applied communication students. 



Table 2 

Pillai's Trace Test for Multivariate Analysis 



Value 


Source 


F 


NDF 


DDF 


PR>F 


Course 


.78878902 


43.3075 


4 


266 


.0001* 


Time 


.01707687 


1.1467 


2 


132 


.3208 


Interaction 


.09904930 


3.4650 


4 


266 


.0088* 



* Significant 



Table 3 

ANOVA for PSI and WGCTA 



PSI 


DF 


SS 


MS 


F 


PR>F 


Course 


2 


9076.396 


4538.198 


7.82 


.0006* 


Error 


133 


77163.659 


580.177 






Time 


1 


111.202 


111.202 


.95 


.3322 


Interaction 


2 


596.763 


298.381 


2.54 


.0825 


Error 


133 


15613.174 


117.392 






R-square=.847 












WGCTA 


DF 


SS 


MS 


F 


PR>F 


Course 


2 


12143.342 


6071.671 


75.67 


.0001* 


Error 


133 


10671.936 


80.240 






Time 


1 


21.208 


21.208 


.83 


.3632 


Interaction 


2 


247.125 


123.562 


4.85 


.0093* 


Error 


133 


3388.153 


25.474 






R-square=.871 













* Significant 



The second null hypothesis, H02, stated there is no statistically significant 
difference in the mean total scores from the PSI and WGCTA by time of administration of 
the test instruments. H02 was not rejected at the .05 level of significance. As reported in 
Table 2, there was no significant difference between the pretest and posttest mean total 
scores from the PSI and WGCTA, for any class of students. 

In addition, results were examined to check for interactions. An analysis of 
variance procedure revealed that there was a significant interaction between the mean 
WGCTA scores by the type of English course assignment and time. A post hoc test (Table 
4) was used to isolate source of the differences between the interaction of course and time. 
It was ascertained that the mean pretest and posttest WGCTA test scores of honors English 
in students were significantly higher than the mean pretest and posttest WGCTA scores of 
English HI and applied communications students. Also, English m student’s pretest and 
posttest scores were significantly higher than the pretest and posttest scores of applied 
communications students. 
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Table 4 

Least Square Mean, Least Square Standard Error and Least Significant 



Difference Test for Dependent Variables PSI and WGCTA by Course 



Course 


n 


M 


SE 




LSD Test 
2 3 


Dependent Variable=PSI 














Honors English HI 


106 


81.367 


1.052 


1 


.0001* 


.0001* 


English m 


86 


91.604 


1.168 


2 




.0904 


English me 


80 


94.475 


1.211 


3 






Dependent Variable=WGCTA 












Honors English m 


106 


58.301 


.490 


1 


.0001* 


.0001* 


English m 


86 


47.058 


.544 


2 




.0001* 


English me 


80 


42.837 


.564 


3 







* Significant 



The third null hypothesis, H03, stated the correlation between the student’s scores 
from the PSI and from the WGCTA is not statistically different from zero. H03 was tested 
using the Pearson Product-Moment calculation. It was ascertained that the correlation of 
.41 for the students’ posttest scores from the PSI and WGCTA was significantly different 
than zero. Therefore, null hypothesis H03 was rejected. 

The results of this study do not support a conclusion that the critical thinking skills 
of any of the three groups of subjects of this study were significantly changed during the 
course of the school year. Therefore, it cannot be concluded that the applied 
communications course significantly changed students’ critical thinking skills. On the 
other hand, it cannot be concluded that the honors English m or the traditional English in 
courses significantly changed students’ critical thinking skills. 

Implications 



These results may support a belief that critical thinking skills and problem solving 
skills cannot be developed in a short period of time. This explanation is consistent with a 
conclusion reached by Langholz and Smaldino (1989). They pointed out that there is not 
much evidence to support the conclusion that critical thinking and problem solving can be 
developed in a short period of time. 

There is evidence from the literature that would support the idea that training can 
develop a more positive self-appraisal of an individual’s problem solving abilities. 
Gallagher (1993) pointed out that the PSI has been used as an outcome measure for 
problem solving seminars. In a study of an eclectic approach to training paraprofessionals 
in counseling, Gallagher administered the PSI both pre and posttest and found that the PSI 
scores of trainees decreased significantly, indicating that the trainees’ self-appraisal of their 
problem solving was more positive after training. However, the self-appraisal of their 
problem solving abilities by the subjects included in the present study did not significantly 
change over the time of the study. In addition, no statistically significant evidence was 



found indicating that the spread between the self-appraisal of Honors English Eli students 
as compared to the self-appraisal of applied communications students increased over time. 

Although the change was not statistically significant, the posttest mean PSI scores 
of Honors English III and English HI students were lower than their pretest mean scores. 
This could indicate that a slightly more positive self-appraisal of their problem solving 
abilities was developed. By way of contrast, the posttest mean PSI score (95.73) of the 
applied communications students was higher than their pretest mean score (93.23). A 
possible explanation for this slightly less positive self-appraisal might be that they 
developed a more realistic awareness of their problem solving abilities during the applied 
communications course. 

Student’s grade point averages and PLAN test scores were also examined to 
provide additional background information for the study. An examination of the data 
revealed that both the grade point average (3.5 vs. 2.2) and PLAN test scores (23.1 vs. 

14.2) of the honors English students were higher than for applied communication students. 
Students’ scores from the PLAN test, which requires higher order thinking skills, were 
significantly related to their PSI and WGCTA scores. This indicates that the students 
assigned to the different courses had different characteristics which likely influenced their 
performance on the test instruments. 

The results of this study indicate that there was no significant improvement in the 
critical thinking abilities of students who complete the applied communications course. It 
further indicates that these students’ perceptions of their problem solving skills does not 
become more positive as a result of completing this course. 

The Tech Prep curriculum for the applied communications course should 
incorporate specific objectives for improving the critical thinking skills of students and for 
developing a more positive perception of their ability to solve unstructured problems of the 
type encountered at home and at work. To achieve these objectives, instruction specifically 
designed to develop inductive and deductive reasoning skills and to develop the ability to 
draw inferences should be incorporated into the Tech Prep applied communications 
curriculum. 

In addition, other activities and courses associated with the Tech Prep initiative 
should be examined to ascertain to what extent they contribute to the development of 
critical thinking skills of students. Such information is needed as we face the challenge of 
better preparing all students to participate in a global and technically oriented work force. 
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